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Background/ Context : 

Out-of-school factors can significantly impact students’ readiness to leam and thrive in 
school. Research spanning decades confirms that larger social structures and contexts beyond 
the school are critical, accounting for up to two-thirds of the variance in student achievement 
(Coleman et al., 1966; Rothstein, 2010; Phillips, Brooks-Gunn, Duncan, Klebanov, & Crane, 
1998). These out-of-school factors can be particularly pernicious in the context of poverty. For 
example, poverty limits families’ ability to invest in children’s growth, creates pervasive stress, 
and exposes children to chaos and environmental contagions (Brooks-Gunn & Duncan, 1997; 
Dearing 2008; Evans, 2004). Many have argued that schools cannot hope to close the 
achievement gap without addressing out-of-school factors (Berliner 2009; Bryk, Sebring, 

Allens worth, Luppescu & Easton 2010). 

Historically, schools have always addressed some of these factors through the work of 
school nurses, counselors, social workers, and psychologists, but this student support work has 
varied widely, without a standardized set of practices (Lean and Colucci, 2010). City Connects 
provides an innovative and systematic approach to addressing out-of-school barriers to 
achievement and thriving in school. Its approach is grounded in best practices for systematizing 
student support work (Marx, Wooley & Northrop 1998; Adelman & Taylor 2006) and is guided 
by contemporary understandings of child development (Masten 2001; Bronfenbrenner & Morris 
1998; Alexander & Entwisle, 1988). Also guiding the City Connects approach is theory on 
schools as developmental contexts for learning to self-regulate emotions, behavior, and cognitive 
processes (Eccles & Roeser 2005). Schools should be most successful when they support 
children in ways that foster achievement motivation, a high sense of efficacy, and internalized 
responsibility. Given high-quality support, students are expected to develop beliefs and attitudes 
that promote effort and academic engagement, leading to increased academic skills and, finally, 
to improved achievement (Deci & Ryan, 2000; Dweck, 1999; Eccles et al, 1993; Kellaghan, 
Madaus & Raczek 1996). Crucially, given this theory, effective student support interventions 
should lead to benefits that persist even after students have left the intervention, because they 
strengthen beliefs and attitudes that underlie achievement. 

City Connects addresses the out-of-school factors by working with every teacher in the 
school to identify each student’s strengths and needs in academic, social-emotional, health, and 
family domains. City Connects staff then work with community agencies and families to deliver 
a tailored set of services to every child. Continuous assessment and follow up helps ensure that 
students’ strengths and needs are being addressed through the school year. 

Purpose / Objective / Research Question / Focus of Study: 

Past research suggests that receiving City Connects in elementary school positively 
affects academic trajectories, both in elementary school and through the transition to middle 
school and early high school. Growth curve analysis showed that City Connects students have 
significantly greater improvement over time in report card outcomes in reading, writing, and 
math during elementary school than comparison students (Center for Optimized Student Support, 
2010). Additionally, students enrolled in City Connects during elementary schools demonstrate 
higher middle school report card and statewide standardized test scores (both in English 
language arts and math) relative to students never enrolled in City Connects (Walsh, Madaus, 
Raczek et al. 2014). City Connects appears to positively impact non-cognitive academic 
outcomes as well: relative to peers never in City Connects, students enrolled in City Connects 
during elementary school have lower rates of chronic absenteeism and being held back in grade 
through middle and early high school grades (Center for Optimized Student Support, 2012). 
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The next stage of this trajectory is completing high school. Reducing high school 
dropout is an important policy goal, as dropout has clear, measurable, negative impacts for both 
individuals and broader society (De Witte & Rogge, 2013; Hanushek & Wobmann, 2007; 
McNeal, 1995; Strom & Boster, 2007). Though the reasons for dropping out are often 
multifaceted (Heers, Van Klaveren, Groot & Massen van den Brink 2011; Rumberger, 2011; 
Thyssen et al., 2010), there is strong evidence that dropout is correlated with elementary and 
middle school outcomes (Balfanz & Herzog, 2005; Balfanz, Herzog, & Mac Iver, 2007; Heppen 
& Therriault, 2008; Wells, Bechard, & Hamby, 1989). Given the positive treatment effect 
estimates on many of these early correlates of dropout, past research on City Connects suggests 
that one approach to reducing high school dropout may be to implement a systematic student 
support system in elementary schools. Unfortunately, past research could not examine this 
question directly because data from students who received City Connects in elementary school 
and had aged through high school were not made available to program evaluators until recently. 
With such data now available, the purpose of this study is to examine this dropout question 
directly: Do students exposed to City Connects during elementary school drop out of high school 
at a lower rate than students who were never exposed to City Connects? 

Setting: 

City Connects is currently implemented in 63 schools in Massachusetts, Ohio, and New 
York. The setting for this study is the Boston Public School (BPS) district, drawing on data from 
school years 2001-02 through 2011-12. In BPS across these years, the intervention was 
implemented in 12 elementary and K-8 schools in several geographic areas in the city. BPS is a 
high-poverty, urban school district. Across all elementary and K-8 schools in the district, about 
90% are students of color, and over 90% are eligible for free or reduced-price lunch. 

Population / Participants / Subjects: 

The analytic sample consists of 2,265 City Connects and 19,979 comparison students (see 
analytic sample inclusion criteria in Data Collection and Analysis below). City Connects and 
comparison samples have similar proportions of males (-51%), English language learners 
(-23%), and students eligible for reduced (-3%) and free lunch (-91%). The two groups differ 
more in race and country of birth (foreign born) - the City Connects sample has a higher 
proportion of Asian students (20% vs 8%), a lower proportion of Black students (31% vs 41%), 
and a higher proportion of students who are foreign bom (20% vs 16%). Full sample description 
can be found in Appendix B, Table 1. 

Intervention / Program / Practice: 

At the core of the intervention is a full-time Site Coordinator, a Masters’ -trained school 
counselor or social worker, who: 1) collaborates with every classroom teacher to identify for 
each student key areas of strengths and needs across academic, social-emotional, health and 
family domains; 2) connects students to a tailored set of community-based support services and 
enrichment opportunities; 3) documents and tracks the service plan for each student; and 4) 
follows up to assure service delivery and assess effectiveness. The model is codified in a 
Practice Manual, with accompanying training and tools, including a proprietary electronic 
database for tracking service referral and delivery. 

The City Connects theory of change is shown in Appendix B, Figure 1. As shown in the 
theory of change, while the implementation of student support is in elementary school, the 
intervention is predicted to lead to improvements in such areas as academic effort, behavior, and 
work habits, which in turn are expected to lead to improved academic achievement, not just in 
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elementary school, but after students have left City Connects and moved on to middle school, 
high school, and beyond. 

Research Design: 

Our outcome measure is aligned with the commonly understood definition of dropout: 
permanent disenrollment from school prior to completion of high school graduation 
requirements. The outcome variable used in this study can be considered a direct measure of 
dropout status and dropout timing (by grade) for each individual. This is in stark contrast to 
group-level proxies of dropout, such as four/five year graduation rates or cohort enrollment 
differences across grades, which are often used to report official dropout statistics (Rumberger, 
1987) and in educational research/program evaluation (see Hammond, Linton, Smink, & Drew, 
2007; ICF International, 2010). This distinction is worth noting. As Rumberger (1987) has 
pointed out, group-based dropout proxies are defined in manners that often need clarification 
because they do not always align with the common understanding of dropout. Moreover, such 
group-based proxies can reflect a number of extraneous factors, the most obvious of which are 
student transfers after 9 th grade and late, but eventual, graduation from high school. 

We utilize a quasi-experimental design to estimate the effect of receiving City Connects 
in elementary school on later high school dropout. Discrete event history analysis (i.e., 
hierarchical logistic regression) is used to model the log-odds of dropout over time, where time- 
points are nested within students. Time is captured by grade with a series of dummy variables at 
the repeated measures level. To account for student mobility during high school years, a series 
of time varying dummy variables capturing the number of high school transfers experienced by 
student i through grade t is also incorporated into the repeated measures level. A random 
intercept model is specified - the repeated measures intercept is allowed to vary as a function of 
student-level variables, including the treatment indictors and control variables (e.g., 
demographics and mobility history prior to high school). Overall, this strategy results in a non- 
parametric time function that yields grade-specific estimates of the log-odds of dropout for City 
Connects and comparison students after adjusting for demographics at the student level and 
mobility at both the student and repeated measures levels. We note that this model form assumes 
that treatment affects the log odds of dropout proportionally across each grade. We tested this 
assumption and found little evidence of a treatment -by-grade interaction. Consequently, the final 
model was reduced to the form described here. 

For the purpose of estimating treatment effects by grade of initial exposure, an initial City 
Connects grade covariate is also incorporated into the student level. Use of this covariate 
assumes treatment effect linearity across initial exposure grades. This assumption was 
examined, and little evidence of a non-linear relationship was found. 

Finally, we note that treatment subgroups are partially aligned with cohort year. Thus, to 
reduce the likelihood that historical trends bias treatment effect estimates, we account for cohort 
year with a series of dummy variables at the student level. 

Data Collection and Analysis: 

Longitudinal data used in this analysis are drawn from the full district dataset made 
available by BPS, and span school years 2001-02 through 201 1-12. The analysis uses BPS high 
school outcome data only from students who were also enrolled in BPS during elementary school 
grades. The K-5 City Connects intervention began implementation in BPS during school year 
2001-02. Data are analyzed for students from six cohorts: those in kindergarten in 1996-97 
through 2002-03. These cohort restrictions are used because only students from these groups 
could potentially have enrolled in a City Connects schools prior to the end of 5 th grade and also 
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have at least one year of high school outcome data by school year 2011-12. Additionally, the 
analysis is performed using data only from students who were enrolled in BPS at the start of high 
school. This restriction facilitates the estimation of cumulative 9 th grade cohort dropout rates 
(i.e., the percentage of students from an entering 9 th grade class who dropped out at any point 
during high school). 

City Connects students are defined as those enrolled in a City Connects BPS school at 
any point during grades K-5. Comparison students are defined as students who were never 
enrolled in a City Connects school but who were enrolled in a non-City Connects BPS school at 
some point prior to the completion of grade 5. 

Findings / Results: 

Relative to students who were never exposed to the intervention, students enrolled in City 
Connects beginning in kindergarten have approximately half the odds of dropping out of high 
school in any given grade (OR = .502, p < .001). By transforming model coefficients into rates, 
then aggregating across grades 9 - 12, we find that comparison students drop out of high school 
at a rate of -15.2%, while City Connects students drop out at a rate of -8.0%. For an entering 9 th 
grade cohort in BPS of typical size (-5000 students), this difference in rates translates into -358 
fewer dropouts over the course of high school. 

Additionally, we find that earlier initial exposure to City Connects is associated with 
more beneficial treatment effects (OR = 1.186, p = .000). Although treatment effects decrease as 
initial exposure grade increases, beneficial treatment effects remain positive and statistically 
significant through initial exposure in grade 3. Treatment effects are no longer statistically 
different from zero when initial exposure occurs in grades 4 or 5. 

Conclusions: 

Preventing dropout is a challenge because students fail to complete high school for a 
myriad of reasons that involve factors inside and outside of schools (Heers, Van Klaveren, 

Groot, & Van den Brink 2011; Rumberger, 2011; Thyssen et al., 2010). Many policy efforts 
targeting dropout focus on the high school years, even though it is clear that the complex factors 
ultimately leading to dropout often manifest and influence students long before they reach high 
school. The results from this study provide evidence that intervening in the elementary school 
years can make a difference. 

We hypothesize that City Connects may be able to positively affect this long-term 
outcome because it addresses both strengths and needs of every student not only in academics, 
but in behavioral/social/emotional growth, health, and family domains, and connects students to 
the particular supports they need during critical periods of development. Thus, the intervention 
provides the flexibility and robustness needed to help address the myriad of factors that affect 
long-term academic trajectories of students. 

Limitations of this study include possible threats to internal validity. These include 
selection bias due to unobserved variables and possible bias in treatment estimates due to 
attrition resulting from City Connects or comparison students transferring out of the BPS district 
prior to high school entry. Additionally, generalizability may be limited to urban school districts 
similar to BPS, and/or resource rich-city contexts (i.e., cities in which a diverse and large variety 
of community service providers are available). 
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Appendix B. 


Table 1. Analytic Sample Characteristics 




Comparison 

CCNX 

Total 



Count 

Col. % 

Count 

Col. % 

Count 

Col. % 


Total 

19979 

100.0% 

2265 

100.0% 

22244 

100.0% 

Cohort Year 

1996 

3268 

16.4% 

227 

10.0% 

3495 

15.7% 


1997 

3227 

16.2% 

281 

12.4% 

3508 

15.8% 


1998 

3127 

15.7% 

353 

15.6% 

3480 

15.6% 


1999 

2955 

14.8% 

318 

14.0% 

3273 

14.7% 


2000 

3006 

15.0% 

333 

14.7% 

3339 

15.0% 


2001 

2652 

13.3% 

326 

14.4% 

2978 

13.4% 


2002 

1744 

8.7% 

427 

18.9% 

2171 

9.8% 

Special 

non SPED 

16144 

80.8% 

1739 

76.8 % 

17883 

80.4% 

Education 

Status 

reg ed with modifications 

14 

.1% 

3 

.1% 

17 

.1% 

reg ed with up to 25 percent 

out 

1188 

5.9% 

166 

7.3% 

1354 

6.1% 


reg ed with up to 60 percent 

out 

1920 

9.6% 

267 

11.8% 

2187 

9.8% 


substantially separate 

713 

3.6% 

90 

4.0% 

803 

3.6% 

ELL 

no 

15513 

77.6% 

1719 

75.9% 

17232 

77.5% 


yes 

4466 

22.4% 

546 

24.1% 

5012 

22.5% 

Free 

full price lunch 

1242 

6.2% 

139 

6.1% 

1381 

6.2% 

Reduced 
Lunch Status 

reduced price lunch 

512 

2.6% 

67 

3.0% 

579 

2.6% 

free lunch 

18225 

91.2% 

2059 

90.9% 

20284 

91.2% 

Gender 

female 

10208 

51.1% 

1155 

51.0% 

11363 

51.1% 


male 

9771 

48.9% 

1110 

49.0% 

10881 

48.9% 

Foreign Born 

no 

16758 

83.9% 

1809 

79.9% 

18567 

83.5% 


yes 

3221 

16.1% 

456 

20.1% 

3677 

16.5% 

Race 

Black 

8118 

40.6% 

707 

31.2% 

8825 

39.7% 


White 

2229 

11.2% 

263 

11.6% 

2492 

11.2% 


Asian 

1671 

8.4% 

452 

20.0% 

2123 

9.5% 


Hispanic 

7748 

38.8% 

826 

36.5% 

8574 

38.5% 


Native American 

72 

.4% 

7 

.3% 

79 

.4% 


Mixed/Other 

141 

.7% 

10 

.4% 

151 

.7% 

Elementary 

1 

2128 

10.7% 

165 

7.3% 

2293 

10.3% 

and Middle 
School 
Mobility 
(number of 

2 " 

9398 

47.0% 

949 

41.9% 

10347 

46.5% 

3 

5438 

27.2% 

667 

29.4% 

6105 

27.4% 

4 

2073 

10.4% 

297 

13.1% 

2370 

10.7% 

school 

transfers) 

5 

942 

4.7% 

187 

8.3% 

1129 

5.1% 
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Figure 1. City Connects theory of change 
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Implements student support as • 
a core function of schools 

Trains staff on a codified 
practice 

Collaborates with each teacher 
to develop a uniquely tailored 
student support plan for every 
child 

Leverages both school- and 
community-based services and 
enrichment opportunities 

Partners with families to 
ensure service delivery 

Systematically uses data to 
improve practice and assess 
outcomes 


Students' strengths and 
needs are addressed 

Teachers shape instruction 
to meet student challenges 

• Principals see improvements in 
school climate 

• Families have a trusted person 
to facilitate access to supports 

• Community Providers can 

better tailor and deliver services 


• Significant improvements in 
student achievement & ability to 
thrive 

• Thriving: Behavior, Work 
Habits, & Effort 

• Academic Performance: 
Elementary School Grades 

i 

• Lasting Positive 
Academic Effects 

• Improved Life Chances 
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